Devices and methods for an imaging system with a dual camera architecture

ABSTRACT

An electronic device can include a first image sensor configured to capture a first image of a field of view and a second image sensor configured to capture a second image of the field of view. The electronic device can include a color filter adjacent to the second image sensor such that the field of view is viewable by the second image sensor through the color filter. The first image can have a first pixel resolution. The second image can have a second pixel resolution. The electronic device can include a controller configured to determine a third image based on luminance content of the first image and color content of the second image. The third image can have a third pixel resolution indicative of a spatial resolution of the first image and a spectral resolution of the second image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/970,887, filed May 4, 2018, which is a continuation of U.S.application Ser. No. 15/265,867, filed Sep. 15, 2016, which is acontinuation of U.S. application Ser. No. 13/960,920, filed Aug. 7,2013. The foregoing applications are incorporated herein by reference asif fully set forth in this application.

BACKGROUND

An electronic device such as a notebook computer, cellular phone,tablet, etc. can include a camera to capture images. In some cases, thecamera may exhibit poor light sensitivity due to a small aperture of thecamera or due to color filters also included in the camera. The colorfilters can also reduce the potential sharpness and resolution of theimage by requiring some form of interpolation to reconstruct the colordata over the entire image.

One solution for this problem could be to employ a larger camera.However, due to industrial design requirements for usability,aesthetics, etc., this solution may not be desirable.

SUMMARY

In one example, a device is provided that comprises a first image sensorconfigured to capture a first image of a field of view of the device.The first image can have a first pixel resolution. The first pixelresolution can be indicative of a spatial resolution of the first image.The first image can include a representation of a luminance content ofthe field of view. The device also comprises a second image sensorconfigured to capture a second image of the field of view of the device.The second image can have a second pixel resolution. The second pixelresolution can be indicative of the spectral resolution of the secondimage. The device also comprises a color filter. The color filter can bearranged such that the field of view is viewable by the second imagesensor through the color filter. The second image can include arepresentation of a color content of the field of view due to the colorfilter. The device also comprises a controller configured to determine athird image of the field of view based on the first image and the secondimage. The third image can be based on luminance content of the firstimage and color content of the second image. The third image can have athird pixel resolution indicative of the spatial resolution of the firstimage and the spectral resolution of the second image.

In another example, a method is provided. The method comprises capturinga first image of a field of view by a first image sensor. The firstimage can have a first pixel resolution. The first pixel resolution canbe indicative of a spatial resolution of the first image. The firstimage can include a representation of a luminance content of the fieldof view. The method further comprises capturing a second image of thefield of view by a second image sensor. The second image can have asecond pixel resolution. The second pixel resolution can be indicativeof a spectral resolution of the second image. The second image caninclude a representation of a color content of the field of view basedon a color filter adjacent to the second image sensor. The color filtercan be arranged such that the field of view is viewable by the secondimage sensor through the color filter. The method further comprisesdetermining a third image based on the first image and the second image.The third image can be based on luminance content of the first image andcolor content of the second image. The third image can have a thirdpixel resolution indicative of the spatial resolution of the first imageand the spectral resolution of the second image.

In another example, a device is provided comprising a means forcapturing a first image of a field of view by a first image sensor. Thefirst image can have a first pixel resolution. The first pixelresolution can be indicative of a spatial resolution of the first image.The first image can include a representation of a luminance content ofthe field of view. The device also comprises a means for capturing asecond image of the field of view by a second image sensor. The secondimage can have a second pixel resolution. The second pixel resolutioncan be indicative of a spectral resolution of the second image. Thesecond image can include a representation of a color content of thefield of view based on a color filter adjacent to the second imagesensor. The color filter can be arranged such that the field of view isviewable by the second image sensor through the color filter. The devicealso comprises a means for determining a third image based on the firstimage and the second image. The third image can be based on luminancecontent of the first image and color content of the second image. Thethird image can have a third pixel resolution indicative of the spatialresolution of the first image and the spectral resolution of the secondimage.

In another example, another method is provided. The method comprisesreceiving a first image of a field of view from a first image sensor.The first image can have a first pixel resolution. The first pixelresolution can be indicative of a spatial resolution of the first image.The first image can include a representation of a luminance content ofthe field of view. The method further comprises receiving a second imageof the field of view from a second image sensor. The second image canhave a second pixel resolution. The second pixel resolution can beindicative of a spectral resolution of the second image. The secondimage can include a representation of a color content of the field ofview based on a color filter adjacent to the second image sensor. Thecolor filter can be arranged such that the field of view is viewable bythe second image sensor through the color filter. The method furthercomprises determining a third image based on the first image and thesecond image. The third image can be based on luminance content of thefirst image and color content of the second image. The third image canhave a third pixel resolution indicative of the spatial resolution ofthe first image and the spectral resolution of the second image.

In another example, a device is provided comprising a means forreceiving a first image of a field of view from a first image sensor.The first image can have a first pixel resolution. The first pixelresolution can be indicative of a spatial resolution of the first image.The first image can include a representation of a luminance content ofthe field of view. The device also comprises a means for receiving asecond image of the field of view from a second image sensor. The secondimage can have a second pixel resolution. The second pixel resolutioncan be indicative of a spectral resolution of the second image. Thesecond image can include a representation of a color content of thefield of view based on a color filter adjacent to the second imagesensor. The color filter can be arranged such that the field of view isviewable by the second image sensor through the color filter. The devicealso comprises a means for determining a third image based on the firstimage and the second image. The third image can be based on luminancecontent of the first image and color content of the second image. Thethird image can have a third pixel resolution indicative of the spatialresolution of the first image and the spectral resolution of the secondimage.

These as well as other aspects, advantages, and alternatives, willbecome apparent to those of ordinary skill in the art by reading thefollowing detailed description, with reference where appropriate to theaccompanying figures.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a block diagram of an example imaging system.

FIG. 2A is a front view of an example device.

FIG. 2B is a back view of the example device shown in FIG. 2A.

FIG. 2C is a side view of the example device shown in FIGS. 2A and 2B.

FIG. 2D is a block diagram of the example device shown in FIG. 2C.

FIG. 3 is a block diagram of an example method for operating an exampledevice, in accordance with at least some embodiments described herein.

FIG. 4 is a block diagram of an example method for operating an exampledevice including two image sensors to align images captured by the twoimage sensors, in accordance with at least some embodiments describedherein.

FIG. 5A illustrates an example first image captured by a first imagesensor in an example device, in accordance with at least someembodiments described herein.

FIG. 5B illustrates an example second image captured by a second imagesensor in the example device described in FIG. 5A, in accordance with atleast some embodiments described herein.

FIG. 5C illustrates an example combined image that includes the firstimage superimposed on the second image described in FIGS. 5A-5B.

FIG. 5D illustrates an example third image determined by the exampledevice described in FIGS. 5A-5C, in accordance with at least someembodiments described herein.

FIG. 6 is a block diagram of an example method for operating a computingdevice, in accordance with at least some embodiments described herein.

FIG. 7 is a block diagram of an example method for operating a computingdevice to determine a three-dimensional image, in accordance with atleast some embodiments described herein.

FIG. 8 depicts an example computer-readable medium configured accordingto at least some embodiments described herein.

DETAILED DESCRIPTION

The following detailed description describes various features andfunctions of the disclosed systems and methods with reference to theaccompanying figures. In the figures, similar symbols identify similarcomponents, unless context dictates otherwise. The illustrative system,device and method embodiments described herein are not meant to belimiting. It may be readily understood by those skilled in the art thatcertain aspects of the disclosed systems, devices and methods can bearranged and combined in a wide variety of different configurations, allof which are contemplated herein.

An electronic device may be utilized to obtain an image of a field ofview of the electronic device. Within examples described herein, adevice, such as a digital camera, computing device, wearable device,mobile device, cellular phone, tablet, etc., including two image sensorsis provided. The device can be configured to provide an image containingluminance content from one image sensor of the two image sensors andcolor content from another image sensor of the two image sensors.

Within a specific example, a device is provided that includes a firstimage sensor and a second image sensor configured, respectively, toobtain a first image having a first pixel resolution and a second imagehaving a second pixel resolution, of a field of view of the device. Thefirst pixel resolution can be indicative of a spatial resolution of thefirst image. The second pixel resolution can be indicative of a spectralresolution of the second image. The first image can include arepresentation of a luminance content of the field of view. The devicecan also include a color filter adjacent to the second image sensor suchthat the field of view is viewable by the second image sensor throughthe color filter. The second image can include a representation of acolor content of the field of view due to the color filter. The devicecan also include a controller configured to determine a third imagebased on luminance content of the first image and color content of thesecond image. The third image can have a third pixel resolutionindicative of the spatial resolution of the first image and the spectralresolution of the second image. The device can optionally include adisplay and the controller can be further configured to provide thethird image to the display.

In some examples, the first pixel resolution of the first image isgreater than the second pixel resolution of the second image. In thisexample, the controller can be configured to provide the third imagehaving the third pixel resolution that is substantially same as thefirst pixel resolution of the first image.

In some examples, the controller can be configured to determineluminance content of the second image. For example, the second image canbe substantially dark (e.g., low luminance content) due to the colorfilter and/or low luminance content reaching the second image sensorfrom the field of view. For example, the luminance content of the secondimage can be compared to a first threshold value corresponding to lowluminance content and, based on the comparison (e.g., luminance contentof the second image is less than the first threshold value), thecontroller can be configured to provide the first image as the thirdimage. In that case, the first image may include significantly morefeatures from the field of view than the second image. Thus, in thisexample, the controller can be configured to provide the first image asthe third image. In this example, the controller provides the firstimage as the third image substantially lacking color content similarlyto human perception of dark environments. Additionally or alternatively,the controller can be configured to determine the color content of thesecond image. For example, the controller can be configured to determinethe second image having substantially low color content (e.g., barcode). For example, the color content of the second image can becompared to a second threshold value corresponding to low color content.In this example, based on the comparison (e.g., the color content beingless than the second threshold value), the controller can also beconfigured to provide the first image as the third image. Additionallyor alternatively, the color content of the second image can be noisy andthus providing the first image as the third image can result in a higherquality third image than a combination of the first image and the secondimage. Thus, in these examples, the device provides a method of usingthe first image as the third image instead of combining the first imageand the second image to determine the third image.

In some examples, the controller can be configured to perform, based ona spatial arrangement of the first image sensor and the second imagesensor, an alignment of features in the first image with correspondingfeatures in the second image to determine the third image.

In some examples, the controller can be configured to use a parallaxeffect due to misalignment between the first image and the second image.In some examples, the controller can be configured to determine depthinformation of the field of view based on the alignment and the spatialarrangement. Additionally or alternatively, the controller can beconfigured to determine a three-dimensional image of the field of viewbased on the third image and the depth information.

In some examples, the controller can be configured, based on a result ofthe alignment, to provide the second image as the third image. Forexample, macrophotography applications requiring a small depth of focusmay result in a substantial misalignment (e.g., unsuccessful result ofthe alignment) between the first image and the second image due to theparallax effect. Thus, in this case, due to the substantial misalignment(e.g., the result of the alignment being unsuccessful), the controllercan be configured to provide the second image as the third image. Inthis example, the third image based on the second image only can includesignificantly higher quality than a combination of the first image andthe second image. Thus, in this example, the device provides a method ofusing the second image as the third image instead of combining the firstimage and the second image to determine the third image.

The device can optionally include a first optical element and a secondoptical element configured to focus light from the field of view,respectively, onto the first image sensor and the second image sensor.In some examples, the first optical element and the second opticalelement can be configured to provide a substantially same depth offield, respectively, to the first image sensor and the second imagesensor. In these examples, the controller can be configured to alignfeatures of the first image with corresponding features of the secondimage based on the substantially same depth of field.

In some examples, the controller can be configured to operate the firstimage sensor and the second image sensor to control the luminancecontent of the first image and luminance content of the second imagebased on focal numbers of the first optical element and the secondoptical element.

Additionally or alternatively, the first optical element and the secondoptical element can be configured to have substantially same focalnumbers. For example, the device can be configured such that theluminance content of the first image is greater than the luminancecontent of the second image based on the substantially same focalnumbers and the color filter adjacent to the second image sensor.

Some embodiments of the present disclosure therefore provide systems andmethods for combining a first image from a first image sensor with asecond image from a second image sensor to determine a third image. Suchdetermined third image can have a spatial resolution substantially sameas a spatial resolution of the first image and a spectral resolutionsubstantially same as a spectral resolution of the second image.

FIG. 1 is a block diagram of an example imaging system 100. In someexamples, the imaging system 100 can be included in a smart phone,digital assistant, digital camera, body-mounted computing device (e.g.,eye-glasses with computing capability), or any other computing deviceconfigured to provide an image of a field of view of the imaging system100. The imaging system 100 can be configured to receive incident light102 from a field of view of the imaging system 100. The incident light102 can be received by a first camera 110 and a second camera 120 thatare both included in the imaging system 100. The first camera 110 andthe second camera 120 can provide information to a controller 130indicative of the incident light 102 from the field of view of theimaging system 100, respectively, via interconnects 104 and 106. In someembodiments, the imaging system 100 can include a display 140 configuredto receive an image from the controller 130 via interconnects 108 fordisplay.

The first camera 110 includes a first image sensor 112 configured tocapture a first image of the field of view of the imaging system 100based on the incident light 102. The first image captured by the firstimage sensor 112 can have a first pixel resolution and can include arepresentation of a luminance content of the field of view of theimaging system 100. Resolution is a measure of detail that an imageholds. For example, spatial resolution can be a measure of how closelylines (e.g., edges) can be resolved in the image (e.g., perceived byperson looking at the image). In another example, spectral resolutioncan be a measure of distinction between light including more than onespectrum (e.g., multiple wavelengths, different colors, etc.). Inexamples including a digital image, resolution can be measured as pixelresolution. Pixel resolution can be a number of pixels per unit area ofthe digital image. In some examples, the first pixel resolution of thefirst image can be indicative of a spatial resolution of the firstimage. For example, the spatial resolution of the first image cancorrespond to resolution of spatial features represented in the incidentlight 102 from the field of view of the imaging system 100.

The first image sensor 112 included in the first camera 110 can include,for example, an array of semiconductor pixel sensors configured toreceive the incident light 102 and provide data indicative of the firstimage of the field of view to the controller 130 via the interconnects104. In other examples, the first image sensor 112 may include acharge-coupled device (CCD). In some examples, the first image sensor112 may be coupled to a printed circuit board and arranged to receivethe incident light 102 and provide the data indicative of the firstimage through the printed circuit board and interconnects 104 to thecontroller 130.

The first camera 110 can optionally include a first optical element 116.The first optical element 116 can be configured to focus incident light102 onto the first image sensor 112 to facilitate determining the firstimage by the first image sensor 112. The first optical element 116 maycomprise one or more lens, mirrors, prisms, filters or any othercomponent configured to process incident light 102 propagating towardsthe first optical element 116 from the field of view of the imagingsystem 100. The first optical element 116 can be configured to direct,reflect, and/or focus the incident light 102 from the field of view ontothe first image sensor 112.

Although not illustrated in FIG. 1 , the first camera 110 can optionallyinclude an actuator coupled to the first optical element 116 andconfigured to cause a change in a position of the first optical element116 to control focus of the first image generated by the first imagesensor 112. In some examples, the actuator can be controlled by thecontroller 130 via interconnects 104. In some examples, the actuator maycomprise a voice coil motor (VCM), a piezoelectric actuator, MEMS, or ashape memory alloy.

The second camera 120 included in the imaging system 100 includes asecond image sensor 122 configured to capture a second image of thefield of view of the imaging system 100 based on the incident light 102.The second image captured by the second image sensor 122 can have asecond pixel resolution. The second camera 120 can include a colorfilter 124 arranged along a path of the incident light 102 such that thefield of view of the imaging system 100 is viewable by the second imagesensor 122 through the color filter 124. The second image can include arepresentation of a color content of the field of view of the imagingsystem 100 due to the color filter 124. The second pixel resolution ofthe second image can be indicative of a spectral resolution (e.g.,resolution of colors, different wavelengths, etc.) of the second image.For example, the spectral resolution of the second image can correspondto resolution of spectral features (e.g., colors) represented inincident light 102 from the field of view of the imaging system 100.

In some examples, the configuration, structure, operation, andarrangement of the second image sensor 122 can be similar to the firstimage sensor 112 included in the first camera 110. In some embodiments,the second camera 120 can include a second optical element 126 similarto the first optical element 116 included in the first camera 110. Insome embodiments, the second camera 120 can include a second actuatorsimilar to the actuator described in the discussion for the first camera110.

The color filter 124 included in the second camera 120 can be arrangedadjacent to the second image sensor 122 such that the incident light 102from the field of view of the imaging system 100 is viewable by thesecond image sensor 122 through the color filter 124. The color filter124 may comprise a Bayer filter, RGBE filter, CYYM filter, CYGM filter,RGBW filter or any other color filter known in the art configured toselectively allow wavelengths of light included in the incident light102 through the color filter 124 to facilitate the second image sensor122 capturing the second image including a representation of colorcontent (e.g., chrominance, chroma data, etc.) of the field of view ofthe imaging system 100. In some examples, the color filter 124 can bemanufactured using color dyes such as color photoresists (e.g.,CMCR101R, CMCR101G, CMCR101B, etc.).

In some examples, the functions of the color filter 124 can be includedin the second image sensor 122. For example, the second image sensor 122can be a charge-coupled device (CCD) configured to generate the secondimage including color content without color filter 124. Thus, in thisexample, the second image sensor 122 and the color filter 124 can be thesame physical component. In other examples, the second optical element126 may include a color filter such that the second optical element 126and the color filter 124 are the same physical component.

The controller 130 can be configured to operate the first camera 110 andthe second camera 120 to obtain data corresponding to the first imageand the second image via interconnects 104 and 106. In some examples,the controller 130 can be included in a computing device. In otherexamples, the controller 130 can be a remote server configured toreceive the data corresponding to the first image and the second image.In some examples, the controller 130 can be configured to determine athird image of the field of view based on the first image and the secondimage. The controller 130 can be configured to determine the third imagebased on luminance content of the first image and color content of thesecond image. The third image can have a third pixel resolutionindicative of the spatial resolution of the first image and the spectralresolution of the second image.

In some examples, the first image sensor 112 and the second image sensor122 can be configured such that the first pixel resolution of the firstimage is greater than the second pixel resolution of the second image.As mentioned earlier, the first pixel resolution can be indicative ofthe spatial resolution of the first image and the second pixelresolution can be indicative of the spectral resolution of the secondimage. In some examples, the spatial resolution can correspond toresolution of spatial features (e.g., lines, edges, etc.) within thefield of view of the imaging system 100. Additionally, in some examples,the spectral resolution can correspond to resolution of spectralfeatures (e.g., colors) within the field of view. Human perception ofspatial resolution is greater than spectral resolution. That is, humansare more sensitive to spatial aspects (e.g., sharpness of edges,separation between lines, etc.) than spectral aspects (e.g., color,chrominance, etc.). Thus, the first image containing luminance content(e.g., luma data, brightness, etc.) with the greater first pixelresolution can be combined via the controller 130 with the second imagecontaining color content (e.g., chrominance, chroma data, etc.) havinglower second pixel resolution to provide the third image having a thirdpixel resolution substantially same as the first pixel resolution. Inthis example, the third image can be perceived having a greaterbrightness, sharpness and resolution than the second image whileincluding the color content of the second image. Thus, in the example ofthe digital image, the third image can have substantially same number ofpixels per unit area (e.g., pixel resolution) as the first image whileincluding color content extracted from the second image having lowerpixels per unit area (e.g., pixel resolution). For example, a viewer ofthe third image may not perceive the lower spectral resolution (e.g.,color content) of the third image due to lower spectral sensitivity ofthe viewer's eyes while perceiving the greater spatial resolution andluminance (e.g., brightness) of the third image due to the higherspatial sensitivity of the viewer's eyes.

In some examples, the controller 130 can be configured to perform analignment of features in the first image with corresponding features inthe second image to determine the third image. For example, thecontroller 130 can determine an edge of an object in the first image anda corresponding edge in the second image. Thus, the controller 130 canalign the first image and the second image based on the edge and thecorresponding edge to determine the third image.

In some examples, the controller 130 can be configured, based on aresult of the alignment, to provide the second image as the third image.For example, macrophotography applications requiring a small depth offocus may result in a substantial misalignment (e.g., unsuccessfulresult of the alignment) between the first image and the second imagedue to a parallax effect. Thus, in this case, due to the substantialmisalignment (e.g., the result of the alignment being unsuccessful), thecontroller 130 can be configured to provide the second image as thethird image. In this example, the third image based on the second imageonly can include significantly higher quality than a combination of thefirst image and the second image.

In some examples, the controller 130 can be configured to determineluminance content of the second image generated by the second imagesensor 122 and provide the first image as the third image based on thedetermined luminance content of the second image. For example, thesecond image can be substantially dark (e.g., low luminance content) dueto the color filter 124 and/or low luminance content reaching the secondimage sensor 122 from the field of view. In that case, the first imagemay include substantially more features from the field of view than thesecond image. For example, the luminance content of the second image canbe compared to a first threshold value corresponding to low luminancecontent. Thus, in this example, based on the luminance content of thesecond image being less than the first threshold value, controller 130can be configured to provide the first image as the third image. In thisexample, the controller 130 can provide the first image as the thirdimage substantially lacking color content similarly to human perceptionof dark environments.

Additionally or alternatively, the controller 130 can be configured todetermine the color content of the second image. For example, thecontroller 130 can be configured to determine the second image havingsubstantially low color content (e.g., bar code). For example, the colorcontent of the second image can be compared to a second threshold value.In this example, based on the color content of the second image beingless than the second threshold value, the controller 130 can also beconfigured to provide the first image as the third image.

In some examples, the controller 130 can be configured to perform, basedon a spatial arrangement of the first image sensor 116 and the secondimage sensor 126, an alignment of features in the first image withcorresponding features in the second image to determine the third image.For example, the first image sensor 116 and the second image sensor 126can be proximally arranged along a same axis a given distance from eachother. Thus, in this example, the controller 130 can be configured toperform the alignment based on the given distance along the same axis.

In some examples, the controller 130 can be configured to use a parallaxeffect due to misalignment between the first image and the second image.For example, the controller 130 can be configured to determine depthinformation of the field of view of the imaging system 100 based on thealignment and the spatial arrangement. For example, features of anobject (e.g., in the field of view) in the first image can be determinedto be at a particular distance from corresponding features of the objectin the second image when the first image is superimposed on the secondimage. Thus, in this example, the controller 130 can determine thedistance between the imaging system 100 and the object based on theparticular distance between the feature and the corresponding feature,and based on the spatial arrangement of the first image sensor 116 andthe second image sensor 126.

Additionally or alternatively, the controller 130 can be configured todetermine a three-dimensional image of the field of view of the imagingsystem 100 based on the third image and the depth information. In theexample of the object described above, the depth information can furtherinclude distance between the imaging system 100 and several features ofthe object. Thus, in this example, based on the depth information andthe third image, the controller 130 can be configured to determinethree-dimensional aspects of the object and provide a three-dimensionalimage of the field of view representing the three-dimensional aspects.

In examples where the imaging system 100 includes a first opticalelement 116 and a second optical element 126 configured to provide asubstantially same depth of field, respectively, to the first imagesensor 112 and the second image sensor 122, the controller 130 can beconfigured to align features of the first image and correspondingfeatures of the second image based on the substantially same depth offield. For example, the first optical element 116 can include one ormore lens configured to focus light form the field of view onto thefirst image sensor 112 to provide a first depth of field to the firstimage sensor 112. Similarly, the second optical element 126 can beconfigured to provide a second depth of field to the second image sensor122. In some examples, the first optical element 116 and the secondoptical element 126 can be configured such that the first depth of fieldis substantially same as the second depth of field. Thus, in theseexamples, the controller 130 can be configured to align features of thefirst image with corresponding features of the second image based on thesubstantially same depth of field.

In some examples, the controller 130 can be configured to operate thefirst image sensor 112 and the second image sensor 122 to control theluminance content of the first image and luminance content of the secondimage based on focal numbers of the first optical element 116 and thesecond optical element 126. Focal number can be a ratio of an opticalelement's focal length to a diameter of the optical element. The focalnumber can be a measure of lens speed. The lens speed can be a measureof luminance content delivered by the optical element per unit time.Thus, in this example, the controller 130 can operate the first imagesensor 112 and the second image sensor 122 to collect light from thefield of view for a given time, based on the focal numbers of the firstoptical element 116 and the second optical element 126, such that thegiven time corresponds to the luminance content of the first image andthe luminance content of the second image.

Additionally or alternatively, the first optical element 116 and thesecond optical element 126 can be configured to have substantially samefocal numbers. For example, the imaging system 100 can be configuredsuch that the luminance content of the first image is greater than theluminance content of the second image based on the substantially samefocal numbers and the color filter 124 adjacent to the second imagesensor 122. In this example, a representation of luminance from thefield of view through the first optical element 116 and the secondoptical element 126 is substantially same due to the substantially samefocal numbers. However, in this example, luminance incident on the firstimage sensor 112 is greater than luminance incident on the second imagesensor 122 due to the color filter 124 adjacent to the second imagesensor 122 filtering a portion of the incident light 102 from the fieldof view.

The controller 130 can optionally include a processor 132 and a memory134. The processor 132 can be configured to receive data indicative ofthe first image and the second image, respectively, from the firstcamera 110 and the second camera 120. The processor 132 can perform thefunctions of the controller 130 described herein to determine the thirdimage based on the first image and the second image. In some examples,the processor 132 can be a computing system that executes softwarestored in the memory 134 to cause the imaging system 100 to process thefirst image and the second image to determine the third image, inaccordance with at least some of the embodiments described herein. Forexample, the memory 134 can include instructions executable by theprocessor 132 to operate the first camera 110 and the second camera 110to capture the first image and the second image.

The imaging system 100 can optionally include a display 140 connected tothe controller 130 via interconnects 108. For example, the imagingsystem 100 can be a smart phone with a touch screen display. In thisexample, the smart phone can show the third image having the luminancecontent of the first image and the color content of the second image onthe touch screen display. The display 140 can include, for example, aliquid crystal display (LCD), an array of light emitting diodes (LED),or any other display known in the art. The display 140 can be configuredto receive data indicative of the third image from the controller 130and provide the third image for display. In some examples, the thirdimage can include a three-dimensional image of the field of view ofimaging system 100 and the display 140 can be configured to display thethree-dimensional image. In some examples, the display 140 can receiveinput from a user indicative of controlling the imaging system 100. Forexample, the display 140 can be a touch screen display configured toreceive an input from a user. In this example, the input can becommunicated to the controller 130, and can cause the controller 130 toinstruct the first camera 110 and the second camera 120 to capture,respectively, the first image and the second image. Thus, in thisexample, the controller 130 can process the first image and the secondimage and provide the third image to the display 140 for display to theuser.

It is noted that the block diagram shown in FIG. 1 is described inconnection with functional modules for convenience in description. Forexample, while the functional blocks in FIG. 1 shown as the first camera110 and the second camera 120 can be implemented by separately packagedcomponents electrically connected to the controller 130, they do notnecessarily need to be implemented as physically separated modules. Theembodiments of the imaging system 100 can be arranged with one or moreof the functional modules (“subsystems”) implemented in a single chip,integrated circuit, and/or physical component. Additionally oralternatively, the color filter 124 and the second image sensor 122 canbe implemented with the same physical device. For example, somecharge-coupled devices (CCD) can both receive incident light 102 anddetermine colors represented in the incident light 102 simultaneously.

FIG. 2A is a front view of an example device 200. FIG. 2B is a back viewof the example device 200 shown in FIG. 2A. It is noted that therelative dimensions in FIGS. 2A and 2B are not necessarily to scale, buthave been rendered for purposes of explanation only in describing anarrangement of the example device 200. The device 200 includes variouselements such as a first camera 210, a second camera 220, a display 240and a body 250. The body 250 includes a first button 252 and a secondbutton 254 configured to receive input for the device 200. Additionallyor alternatively, the display 240 can receive input for the device 200.For example, the display 240 can be a touch screen display configured toreceive input when a user touches the display 240. For illustrationpurposes, an x-y-z axis is shown in the illustrations of FIGS. 2A and2B. The “Front View” shown in FIG. 2A corresponds to the side of thedevice 200 including the display 240 and the second button 254 viewablealong the z-axis pointing out of the page as illustrated in FIG. 2A. The“Back View” shown in FIG. 2B corresponds to the side of the device 200including the first camera 210 and the second camera 220 viewable alongthe z-axis pointing into the page as illustrated in FIG. 2B. Althoughthe device 200 is depicted in FIGS. 2A-2B as a tablet computer, otherembodiments are possible. For instance, the device 200 can be a digitalcamera, smartphone, wearable computer, or a laptop computer, among otherexamples.

The functions performed by elements included in the device 200 such asthe first camera 210, the second camera 220, and the display 240 can besimilar to the functions described, respectively, for the first camera110, the second camera 120, and the display 140 in the discussion ofFIG. 1 . For example, the first camera 210 can be configured to capturea first image having a first pixel resolution and including arepresentation if a luminance content of the field of view of the device200, and the second camera 220 can be configured to capture a secondimage having a second pixel resolution and including a representation ofa color content of the field of view of the device 200. Similarly to thediscussion in FIG. 1 , the first pixel resolution can be indicative of aspatial resolution of the first image, and the second pixel resolutioncan be indicative of a spectral resolution of the second image.Additionally, the display 240 can be configured to display a third imagedetermined by the device 200 based on the first image and the secondimage. The third image can have a third pixel resolution indicative ofthe spatial resolution of the first image and the spectral resolution ofthe second image.

The first camera 210 and the second camera 220 are disposed on a sameside (“Back View”) facing a field of view of the device 200 as shown inFIG. 2B. In some examples, the first camera 210 and the second camera220 can be disposed on any side (e.g., “Front View”, etc.) of the device200 such that both the first camera 210 and the second camera 220 arefacing a same field of view of the device 200. In some examples, thefirst camera 210 and the second camera 220 can be arranged along a sameaxis. For example, in FIG. 2B, the first camera 210 and the secondcamera 220 are arranged along the y-axis. In other examples, the firstcamera 210 and the second camera 220 can be arranged along a differentaxis (e.g., x-axis, etc.). In some examples, the same axis correspondsto an axis along a side of the first image and an axis along acorresponding side of the second image. For example, the third imagedetermined based on the first image and the second image can be providedto display 240 such that the axis along a side of the third image (e.g.,y-axis in FIG. 2A) is substantially same as the axis along which thefirst camera 210 and the second camera 220 are arranged (e.g., y-axis inFIG. 2B).

FIG. 2C is a side view of the example device 200 shown in FIGS. 2A and2B. Incident light 202 from the field of view of the device 200 isincident upon both the first camera 210 and the second camera 220, inaccordance with the embodiments described herein similarly to incidentlight 102 described in the discussion of FIG. 1 . The “Side View” shownin FIG. 2C corresponds to the side of the device 200 including the firstbutton 252 viewable along the x-axis pointing out of the page asillustrated in FIG. 2C.

FIG. 2D is a block diagram of the example device 200 shown in FIG. 2C.The functions and configuration of some elements of the device 200 shownin FIG. 2D such as the first camera 210, the second camera 220, thecontroller 230, interconnects 204, 206 and 208, is similar,respectively, to the first camera 110, the second camera 120, thecontroller 130, interconnects 104, 106, and 108 described in thediscussion of FIG. 1 . For example, the controller 230 can be configuredto receive data indicative of the first image and the second image,respectively, from the first camera 210 and the second camera 220, viainterconnects 204 and 206. In this example, the controller 230 can beconfigured to determine the third image based on the first image and thesecond image and provide the third image for display via interconnects208 to the display 240.

The first camera 210 can include a first image sensor 212 and a firstoptical element 216. The incident light 202 from the field of view ofthe device 200 can propagate through the first optical element 216 ontothe first image sensor 212. The first image sensor 212 can be configuredto capture the first image and provide data indicative of the firstimage to the controller 230 via interconnects 204. The first opticalelement 216 can be similar to the first optical element 116 included inthe imaging system 100, in accordance with the discussion of FIG. 1 .For example, the optical element 116 can include one or more lens,prisms, mirrors or any other optical element configured to focus and/orselectively direct incident light 202 onto the first image sensor 212.The first image generated by the first image sensor 212 can include arepresentation of luminance content (e.g., brightness, luma data, etc.)in the field of view of the device 200.

The second camera 220 can include a second image sensor 222 and a secondoptical element 226 similarly to the second image sensor 122 and thesecond optical element 126 included in the discussion of imaging system100 in FIG. 1 . Additionally, the second camera 220 can include a colorfilter 224 similar to the color filter 124 in the discussion of FIG. 1 .The color filter 124 can be arranged adjacent to the second image sensor222 and along a path of the incident light 202. The second image sensor222 can be configured to generate the second image including arepresentation of a color content (e.g., chrominance, chroma data,colors, etc.) of the field of view of device 200 based on the colorfilter 224. The second image sensor 222 can be configured to providedata indicative of the second image to the controller 230 viainterconnects 206.

Although illustrated in FIG. 2D that the first optical element 216 andthe second optical element 226 are arranged along a back surface 256(e.g., along the “Back View” illustrated in FIG. 2B) of the body 250, insome examples, the first optical element 216 and the second opticalelement 226 can be arranged anywhere along the path of the incidentlight 202 such that the incident light 202 is focused and/or directed bythe first optical element 216 and the second optical element 226,respectively, onto the first image sensor 212 and the second imagesensor 222. Additionally or alternatively, one or more protectivescreens can be disposed along the back surface 256 and configured toprotect the first optical element 216 and the second optical element 226(e.g., from scratches, etc.).

Although not illustrated in FIG. 2D, the first camera 210 can optionallyinclude an actuator coupled to the first optical element 216. In someexamples, the actuator can include a voice coil motor (VCM),piezoelectric actuator, MEMS, or a shape memory alloy. The actuator canbe controlled by the controller 230 and configured to change theposition of the first optical element 216 to change the focus and/ordepth of field of the first image generated by the first image sensor212. In some examples, a second actuator can similarly be included inthe second camera 220 to change the focus and/or depth of field of thesecond image generated by the second image sensor 222.

The controller 230 can be configured to operate the first camera 210 andthe second camera 220, respectively, via interconnects 204 and 206, inaccordance with at least some of the embodiments described herein. Insome examples, the controller 230 can be configured to cause the firstimage sensor 212 to capture the first image and the second image sensor222 to capture the second image. The first image can have a first pixelresolution indicative of a spatial resolution of the first image and thesecond image can have a second pixel resolution indicative of a spectralresolution of the second image. The spatial resolution can be indicativeof resolution of spatial features (e.g., lines, edges, etc.) in thefield of view of the device 200 and the spectral resolution can beindicative of resolution of spectral features (e.g., colors, wavelengthsof incident light 202, etc.) in the field of view of the device 200. Insome examples, the controller 230 can be configured to determine thethird image based on the first image and the second image.

The third image determined by the controller 230 can include luminancecontent of the first image and color content of the second image. Asillustrated in FIG. 2D, the second image generated by the second imagesensor 222 can include less luminance content than the first imagegenerated by the first image sensor 212 due to light filtered by thecolor filter 224. Thus, including the luminance content of the firstimage in the third image can provide better luminance (e.g., brightness)in the third image than the second image. Additionally, the third imagecan have a third pixel resolution indicative of the spatial resolutionof the first image and the spectral resolution of the second image.

In some examples, the controller 230 can be configured to perform analignment of features in the first image with corresponding features inthe second image to determine the third image. In some examples, thealignment can be based on a spatial arrangement of the first camera 210and the second camera 220. For example, the first camera 210 and thesecond camera 220 can be arranged along a same axis. In this example,the same axis can correspond to an axis of a side of the first image andan axis of a corresponding side of the second image. For example, thefirst image and the second image can each have a width side and a heightside. Thus, in some examples, the same axis can correspond to the widthside of the first image and the width side of the second image. In thiscase, the alignment can be performed along the same axis (e.g., widthside) to determine the third image. Such an arrangement can simplify thealignment performed by the controller 230 into an alignment along oneaxis (e.g., the same axis) rather than two axes.

Although illustrated in FIG. 2D that the controller 230 is arrangedbetween the first camera 210 and the second camera 220, the controller230 can be disposed anywhere within the body 250 of the device 200. Forexample, the controller 230 can be placed below the first camera 210 andthe second camera 220.

In some examples, interconnects 204 and 206 can connect the controller230, respectively, to the first camera 210 and the second camera 220.Interconnects 208 can connect the controller 230 to other componentsincluded in the device 200. For example, interconnects 208 can connectthe controller 230 to the display 240, the first button 252, and/or thesecond button 254. In some examples, the controller 230 can beconfigured to receive input, via interconnects 208, indicative ofoperating the first camera 210, the second camera 220, and/or thedisplay 240. For example, a user of the device 200 can touch the display240 to instruct the controller 230 to operate the first camera 210 andthe second camera 220 to capture the first image and the second image.In this example, the controller 230 can also determine the third imageand provide the third image to the display 240 for display.

FIG. 3 is a block diagram of an example method 300 for operating anexample device, in accordance with at least some embodiments describedherein. Method 300 shown in FIG. 3 presents an embodiment of a methodthat could be used with the imaging system 100 and the device 200, forexample. Method 300 may include one or more operations, functions, oractions as illustrated by one or more of blocks 302-306. Although theblocks are illustrated in a sequential order, these blocks may in someinstances be performed in parallel, and/or in a different order thanthose described herein. Also, the various blocks may be combined intofewer blocks, divided into additional blocks, and/or removed based uponthe desired implementation.

In addition, for the method 300 and other processes and methodsdisclosed herein, the flowchart shows functionality and operation of onepossible implementation of present embodiments. In this regard, eachblock may represent a module, a segment, or a portion of a manufacturingor operation process.

At block 302, the method 300 includes capturing a first image of a fieldof view by a first image sensor, wherein the first image has a firstpixel resolution and includes a representation of a luminance content ofthe field of view, wherein the first pixel resolution is indicative of aspatial resolution of the first image.

At block 304, the method 300 includes capturing a second image of thefield of view by a second image sensor, wherein the second image has asecond pixel resolution and includes a representation of a color contentof the field of view based on a color filter adjacent to the secondimage sensor such that the field of view is viewable by the second imagesensor through the color filter, wherein the second pixel resolution isindicative of a spectral resolution of the second image.

At block 306, the method 300 includes determining, based on the firstimage and the second image, a third image based on luminance content ofthe first image and color content of the second image, wherein the thirdimage has a third pixel resolution indicative of the spatial resolutionof the first image and the spectral resolution of the second image.

In some examples, the first pixel resolution can be greater than thesecond pixel resolution. In these examples, the third pixel resolutionof the third image can be substantially same as the first pixelresolution of the first image. For example, (step 302) the first imagecaptured by the first image sensor can be a black & white (e.g.,grayscale) image including a representation of luminance content in thefield of view (e.g., brightness, luma data, etc.). In this example, thesecond image captured by the second image sensor (step 304) can be acolor image due to a color filter adjacent to the second image sensor,the color image including a representation of a color content of thefield of view. Thus, (step 306) the third image can be determined havingluminance data (e.g., luma data) from the first image and color data(e.g., chroma data) from the second image. The third image can have thethird pixel resolution substantially same as the first pixel resolutionwhile incorporating color data of the second image. Since humanperception of spatial resolution (e.g., lines, edges, etc.) is moresensitive than spectral resolution (e.g., colors), a viewer of the thirdimage will perceive the superior spatial resolution and luminance (e.g.,brightness) of the first image and the spectral resolution and colorcontent of the second image.

FIG. 4 is a block diagram of an example method 400 for operating anexample device including two image sensors to align images captured bythe two image sensors, in accordance with at least some embodimentsdescribed herein. Method 400 shown in FIG. 4 presents an embodiment of amethod that could be used with the imaging system 100 and/or the device200, for example. Method 400 may include one or more operations,functions, or actions as illustrated by one or more of blocks 402-408.Although the blocks are illustrated in a sequential order, these blocksmay in some instances be performed in parallel, and/or in a differentorder than those described herein. Also, the various blocks may becombined into fewer blocks, divided into additional blocks, and/orremoved based upon the desired implementation.

In addition, for the method 400 and other processes and methodsdisclosed herein, the flowchart shows functionality and operation of onepossible implementation of present embodiments. In this regard, eachblock may represent a module, a segment, or a portion of a manufacturingor operation process.

At block 402, the method 400 includes capturing a first image of a fieldof view by a first image sensor, wherein the first image has a firstpixel resolution and includes a representation of a luminance content ofthe field of view.

At block 404, the method 400 includes capturing a second image of thefield of view by a second image sensor, wherein the second image has asecond pixel resolution and includes a representation of a color contentof the field of view.

At block 406, the method 400 includes aligning, based on a spatialarrangement of the first image sensor and the second image sensor,features in the first image with corresponding features in the secondimage.

At block 408, the method 400 includes determining, based on thealignment, a third image having luminance content and pixel resolutionof the first image, and color content of the second image.

For example, a computing device (e.g., smartphone, digital camera,tablet, etc.) can include a first image sensor and a second imagesensor. The computing device can cause the first image sensor to capturea first image of a dog in a field of view of the computing device (step402). The first image can have a first pixel resolution and include arepresentation of a luminance content (e.g., brightness of fur) of thedog. The computing device can also cause the second image sensor tocapture a second image of the field of view (step 404) that has a secondpixel resolution and includes a color content of the field of view(e.g., color of the fur of the dog). The computing device can align,based on the spatial arrangement of the first image sensor and thesecond image sensor, features in the first image with correspondingfeatures of the second image (step 406). For example, a nose of the dogin the first image can be aligned with the nose of the dog in the secondimage. The computing device can then determine a third image of the dog(step 408), based on the alignment of the noses, that includes thebrightness of the fur from the first image and the color of the fur fromthe second image.

FIG. 5A illustrates an example first image 500 captured by a first imagesensor in an example device, in accordance with at least someembodiments described herein. The first image 500 of a field of view canbe generated by the first image sensor with a first pixel resolutionsimilar to the first image sensors 112 and 212 described, respectively,in the imaging system 100 and the device 200, for example. The firstpixel resolution can be indicative of a spatial resolution of the firstimage (e.g., sharpness of feature 504). The first image 500 asillustrated in FIG. 5A only contains black lines and white backgroundfor illustrative purposes. However, in some embodiments, the first image500 can include gray colors similar to a grayscale image correspondingto areas in the field of view of the first image sensor that includecolors. Additionally or alternatively, the black lines illustratingedges in the image 500 can include various shades, intensities, etc. ofgray similarly to a grayscale image. Additionally, for illustrativepurposes, the background of the first image 500 is illustrated in awhite color. However, in some embodiments, the background color caninclude varying shades of gray similarly to a grayscale image.

The first image 500, as illustrated, includes hair 502 having no colorcontent. For example, the hair 502 is illustrated in a white color toindicate no color content is included in the first image 500. Otherfeatures such as eyes included in the first image 500 are also lackingcolor content due to the configuration of the first image sensor beingsimilar to the first image sensors 112 and 212 discussed in FIGS. 1-2 .The first image 500 includes a feature 504 (e.g., edge of hair) in thefirst image 500. The feature 504 can be used to align the first image500 with a second image, in accordance with at least some embodimentsdescribed herein.

FIG. 5B illustrates an example second image 510 captured by a secondimage sensor in the example device described in FIG. 5A, in accordancewith at least some embodiments described herein. The second image 510 ofthe field of view can be generated by the second image sensor having asecond pixel resolution similarly to the second image sensors 122 and222 described, respectively, in the imaging system 100 and the device200, for example. The second pixel resolution can be indicative of aspectral resolution of the second image (e.g., colors included in hair,eyes, etc.). The second image 510 as illustrated in FIG. 5B onlycontains gray colors for illustrative purposes. However, in someembodiments, the second image 510 can include colors (e.g., red, green,blue, yellow, brown, etc.) corresponding to various features of thesecond image 510. For example, hair 512 included in the second image 510can be yellow (illustrated as dark gray color in FIG. 5B). Additionallyor alternatively, in some examples, features such as the hair 512 caninclude more than one color (e.g., highlights).

Notably, in this example, the second pixel resolution of the secondimage 510 is lower than the first pixel resolution of the first image500. For example, the feature 504 (e.g., edge of hair) included in thefirst image 500 is sharper than corresponding feature 514 (e.g.,corresponding edge of hair) included in the second image 510.Additionally, the second image 510 captured by the second image sensorcontains less luminance content (e.g., brightness) than the first image500. The lower luminance content of the second image 510 is illustratedin FIG. 5B by the generally darker features such as background 516(e.g., illustrated as a shade of gray) of the second image 510 comparedto corresponding features in the first image 500 shown in FIG. 5A.

FIG. 5C illustrates a combined image 520 that includes the first image500 superimposed on the second image 510 described in FIGS. 5A-5B toillustrate a misalignment between the first image 500 and the secondimage 510, in accordance with at least some embodiments describedherein. For example, the feature 504 and the corresponding feature 514are spatially misaligned in the combined image 520. In some examples,the misalignment can be due to a spatial arrangement of the first imagesensor and the second image sensor, similarly to the discussion in FIGS.1-2 . In other examples, the misalignment can be due to an inclinationangle of the device including the first image sensor and the secondimage sensor relative to the feature 504 and the corresponding feature514.

In some examples, the device configured to capture the first image 500and the second image 510 can also be configured to correct themisalignment based on the feature 504 and the corresponding feature 514.For example, imaging system 100, device 200, and methods 300-400described in the present disclosure describe a process for correctingsuch misalignment based on the feature 504 and the corresponding feature514. In some examples, correcting the misalignment can be based on morethan one feature (e.g., several lines, several edges, etc.) of the firstimage 500 with corresponding features in the second image 510. In someexamples, a controller can be included in the device similar to thecontrollers 130 and 230 included, respectively, in the imaging system100 and the device 200 described in FIGS. 1-2 .

FIG. 5D illustrates an example third image 530 determined by the exampledevice described in FIGS. 5A-5C, based on luminance content of the firstimage 500, color content of the second image 510, and alignment ofcombined image 520, shown in FIGS. 5A-5C, in accordance with at leastsome embodiments described herein. In some examples, the third image 530of the field of view can be generated by a controller to have a thirdpixel resolution similarly to the controllers 130 and 230 described,respectively, in the imaging system 100 and the device 200 using aprocess similar to the process described in methods 300-400, forexample. The third image 530 can be similar to the third image describedin accordance with FIGS. 1-4 included in the present disclosure.

Notably, the third image 530 includes the luminance content of the firstimage 500. For example, the background of the third image 530 isbrighter than background 516 of the second image 510 similarly to thebackground of the first image 500. Additionally, in this example, thecolor of hair 532 included in the third image 530 is brighter than thecolor of hair 512 included in the second image 512 (e.g., illustrated asa brighter shade of gray). For example, the color of hair 512 in thesecond image 510 can be dark green, and the color of hair 532 in thethird image 530 can be a brighter green. Thus, the third image 530includes luminance content of the first image 500 and color content ofthe second image 510.

Additionally, the third image 530 illustrates a successful alignment offeatures included in the first image 500 with corresponding features inthe second image 510. Such a misalignment is illustrated in the combinedimage 520, for example, in feature 504 and corresponding feature 514. Inthe third image, the feature 504 in the first image 500 and thecorresponding feature 514 in the second image 510 are combined asillustrated into combined feature 534.

The third image 530, as illustrated in FIG. 5D, has a third pixelresolution substantially same as the first pixel resolution of the firstimage 500. For example, the feature 504 (e.g., edge of hair) included inthe first image 500 and the combined feature 534 in the third image 530illustrate substantially same sharpness. Thus, in this example, thethird image 530 includes the spatial resolution of the first image 500and the spectral resolution of the second image 510. Since humanperception of spatial resolution is more sensitive than spectralresolution, the third image appears as having the superior quality(e.g., spatial resolution) of the first image 500 while including thecolor content of the second image 510.

FIG. 6 is a block diagram of an example method 600 for operating acomputing device, in accordance with at least some embodiments describedherein. Method 600 shown in FIG. 6 presents an embodiment of a methodthat could be used with the imaging system 100 and/or the device 200,for example. Method 600 may include one or more operations, functions,or actions as illustrated by one or more of blocks 602-604. Although theblocks are illustrated in a sequential order, these blocks may in someinstances be performed in parallel, and/or in a different order thanthose described herein. Also, the various blocks may be combined intofewer blocks, divided into additional blocks, and/or removed based uponthe desired implementation.

In addition, for the method 600 and other processes and methodsdisclosed herein, the flowchart shows functionality and operation of onepossible implementation of present embodiments. In this regard, eachblock may represent a module, a segment, or a portion of a manufacturingor operation process.

At block 602, the method 600 includes receiving a first image of a fieldof view from a first image sensor, wherein the first image has a firstpixel resolution and includes a representation of a luminance content ofthe field of view, wherein the first pixel resolution is indicative of aspatial resolution of the first image.

At block 604, the method 600 includes receiving a second image of afield of view from a second image sensor, wherein the second imagesensor has a second pixel resolution and includes a representation of acolor content of the field of view based on a color filter adjacent tothe second image sensor such that the field of view is viewable by thesecond image sensor through the color filter, wherein the second pixelresolution is indicative of a spectral resolution of the second image.

At block 606, the method 600 includes determining, based on the firstimage and the second image, a third image based on luminance content ofthe first image and color content of the second image, wherein the thirdimage has a third pixel resolution indicative of the spatial resolutionof the first image and the spectral resolution of the second image.

In some examples, a computing device such as a personal computer orserver computer can be configured to perform the method 600. In otherexamples, a processor similar to the processor 132 included in thediscussion of FIG. 1 can be configured to perform the method 600.

For example, (steps 602 and 604) a computing device can receive dataindicative of a first image and a second image captured, respectively,by a first image sensor and a second image sensor included in an imagingdevice (e.g., camera, smartphone, tablet, etc.). The first image canhave a first pixel resolution indicative of a spatial resolution (e.g.,edges, lines, etc.) of the first image and the second image can have asecond pixel resolution indicative of a spectral resolution (e.g.,colors, chrominance, etc.) of the second image. The computing device canbe configured to determine a third image based on luminance content ofthe first image and color content of the second image (step 606). Thethird image can have a third pixel resolution indicative of the spatialresolution of the first image and the spectral resolution of the secondimage. In this example, the computing device can optionally communicatedata indicative of the third image back to the imaging device fordisplay.

FIG. 7 is a block diagram of an example method 700 for operating acomputing device to determine a three-dimensional image, in accordancewith at least some embodiments described herein. Method 700 shown inFIG. 7 presents an embodiment of a method that could be used with theimaging system 100 and/or the device 200, for example. Method 700 mayinclude one or more operations, functions, or actions as illustrated byone or more of blocks 702-710. Although the blocks are illustrated in asequential order, these blocks may in some instances be performed inparallel, and/or in a different order than those described herein. Also,the various blocks may be combined into fewer blocks, divided intoadditional blocks, and/or removed based upon the desired implementation.

In addition, for the method 700 and other processes and methodsdisclosed herein, the flowchart shows functionality and operation of onepossible implementation of present embodiments. In this regard, eachblock may represent a module, a segment, or a portion of a manufacturingor operation process.

At block 702, the method 700 includes receiving a first image of a fieldof view from a first image sensor, wherein the first image has a firstpixel resolution and includes a representation of a luminance content ofthe field of view.

At block 704, the method 700 includes receiving a second image of thefield of view from a second image sensor, wherein the second image has asecond pixel resolution and includes a representation of a color contentof the field of view.

At block 706, the method 700 includes performing an alignment offeatures in the first image with corresponding features in the secondimage to determine a third image.

At block 708, the method 700 includes determining, based on thealignment, depth information of the field of view.

At block 710, the method 700 includes determining, based on the depthinformation, the third image that comprises a three-dimensional image ofthe field of view.

For example, a computing device such as processor 132 described in FIG.1 can be configured to receive a first image from a first image sensorof a field of view of the computing device (step 702). The computingdevice can also be configured to receive a second image from a secondimage sensor of the field of view of the computing device (step 704).The computing device can be configured to perform an alignment (step706) of features in the first image with corresponding features in thesecond image similarly to the alignment described in the discussion ofFIGS. 5A-5D of the present disclosure. Based on the alignment, (step708) the computing device can determine depth information of the fieldof view of the computing device. For example, the computing device candetermine a first distance between corresponding features in the firstimage and the second image when the first image is superimposed on thesecond image (e.g., similarly to FIG. 5C). Additionally, the computingdevice can receive information indicative of a second distance betweenthe first image sensor and the second image sensor. Based on the firstdistance and the second distance, the computing device can determine athird distance (e.g., depth information) between the computing deviceand the feature in the field of view. In some examples, the computingdevice can repeat this procedure with other features in the field ofview to determine a third image comprising a three-dimensional image ofthe field of view (step 710).

FIG. 8 depicts an example computer-readable medium configured accordingto at least some embodiments described herein. In example embodiments,the example system can include one or more processors, one or more formsof memory, one or more input devices/interfaces, one or more outputdevices/interfaces, and machine readable instructions that when executedby the one or more processors cause the system to carry out the variousfunctions tasks, capabilities, etc., described above.

As noted above, in some embodiments, the disclosed techniques (e.g.methods 300, 400, 600, and 700) can be implemented by computer programinstructions encoded on a non-transitory computer readable storage mediain a machine-readable format, or on other non-transitory media orarticles of manufacture (e.g., the instructions stored on the memory 134of the controller 130 of the imaging system 100). FIG. 8 is a schematicillustrating a conceptual partial view of an example computer programproduct that includes a computer program for executing a computerprocess on a computing device, arranged according to at least someembodiments disclosed herein.

In one embodiment, the example computer program product 800 is providedusing a signal bearing medium 802. The signal bearing medium 802 mayinclude one or more programming instructions 804 that, when executed byone or more processors may provide functionality or portions of thefunctionality described above with respect to FIGS. 1-7 . In someexamples, the signal bearing medium 802 can be a non-transitorycomputer-readable medium 806, such as, but not limited to, a hard diskdrive, a Compact Disc (CD), a Digital Video Disk (DVD), a digital tape,memory, etc. In some implementations, the signal bearing medium 802 canbe a computer recordable medium 808, such as, but not limited to,memory, read/write (R/W) CDs, R/W DVDs, etc. In some implementations,the signal bearing medium 802 can be a communication medium 810 (e.g., afiber optic cable, a waveguide, a wired communications link, a wirelesscommunication link, etc.). Thus, for example, the signal bearing medium802 can be conveyed by a wireless form of the communications medium 810.

The one or more programming instructions 804 can be, for example,computer executable and/or logic implemented instructions. In someexamples, a computing device such as the processor-equipped controllers130 and 230 of FIGS. 1 and 2 is configured to provide variousoperations, functions, or actions in response to the programminginstructions 804 conveyed to the computing device by one or more of thecomputer readable medium 806, the computer recordable medium 808, and/orthe communications medium 810. In other examples, the computing devicecan be an external device such as a server or personal computer incommunication with an imaging device such as imaging system 100 ordevice 200.

The non-transitory computer readable medium 806 can also be distributedamong multiple data storage elements, which could be remotely locatedfrom each other. The computing device that executes some or all of thestored instructions could be an external computer, or a mobile computingplatform, such as a smartphone, tablet device, personal computer,head-mounted device, etc. Alternatively, the computing device thatexecutes some or all of the stored instructions could be remotelylocated computer system, such as a server. For example, the computerprogram product 800 can implement the functionalities discussed in thedescription of FIGS. 1-7 .

Within examples, operation methods that are described for the device canbe applied to other electronic devices that include two image sensors.For example, telescopes, microscopes and other optical instruments thatare configured to provide an image of a field of view can be equippedwith two image sensors configured similarly to the devices and methodsdescribed in the present disclosure. Thus, example methods hereinprovide operation methods that involve a device including a first imagesensor configured to capture a first image, a second image sensorconfigured to capture a second image and determining a third image basedon the first image and the second image having a spatial resolution andluminance content of the first image and a spectral resolution and colorcontent of the second image.

It should be understood that arrangements described herein are forpurposes of example only. As such, those skilled in the art willappreciate that other arrangements and other elements (e.g. machines,interfaces, functions, orders, and groupings of functions, etc.) can beused instead, and some elements may be omitted altogether according tothe desired results. Further, many of the elements that are describedare functional entities that may be implemented as discrete ordistributed components or in conjunction with other components, in anysuitable combination and location, or other structural elementsdescribed as independent structures may be combined.

While various aspects and embodiments have been disclosed herein, otheraspects and embodiments will be apparent to those skilled in the art.The various aspects and embodiments disclosed herein are for purposes ofillustration and are not intended to be limiting, with the true scopebeing indicated by the following claims, along with the full scope ofequivalents to which such claims are entitled. It is also to beunderstood that the terminology used herein is for the purpose ofdescribing particular embodiments only, and is not intended to belimiting.

What is claimed is:
 1. A device comprising: a first camera thatcaptures, via a first aperture at a first location of the device, afirst image of a field of view, wherein the first image is a grayscaleimage; a second camera that captures, via a second aperture at a secondlocation of the device, a second image of the field of view, wherein thesecond camera comprises a color filter configured to selectively allowdifferent wavelengths of light received via the second aperture toilluminate respective different sets of pixels of the second camera,wherein the second image is a color image including a representation ofa color content of the field of view, and wherein the first location andsecond location are separated by a distance such that the contents ofthe first image and second image differ due to the parallax effect; adisplay; and a controller, coupled to the first camera and the secondcamera, wherein the controller is configured to: determine a luminancecontent of the second image; based on the luminance content of thesecond image being greater than a threshold, incorporate the colorcontent obtained from the color image into the grayscale image obtainedfrom the first camera to obtain a third image; and cause the display todisplay the third image.
 2. The device of claim 1, further comprising: afirst lens that focuses light from the field of view into the firstcamera; and a second lens that focuses light from the field of view intothe second camera.
 3. The device of claim 2, further comprising: a firstimage sensor included in the first camera, wherein the first imagesensor receives at least a portion of the light focused by the firstlens; and a second image sensor included in the second camera, whereinthe second image sensor receives at least a portion of the light focusedby the second lens.
 4. The device of claim 1, wherein the color filterincludes at least one of a Bayer filter, an RGBE filter, a CYYM filter,a CYGM filter, or an RGBW filter.
 5. The device of claim 1, wherein thegreyscale image includes a representation of luminance content in thefield of view, and wherein the third image includes luminance content ofthe first image and color content of the second image.
 6. The device ofclaim 1, wherein the first image has a first pixel resolution indicativeof a spatial resolution of the first image, and wherein the second imagehas a second pixel resolution indicative of a spectral resolution of thesecond image.
 7. The device of claim 6, wherein the third image has athird pixel resolution indicative of the spatial resolution of the firstimage and the spectral resolution of the second image.
 8. The device ofclaim 6, wherein the first pixel resolution of the first image isgreater than the second pixel resolution of the second image.
 9. Thedevice of claim 8, wherein the third image has a third pixel resolutionthat is substantially same as the first pixel resolution of the firstimage.
 10. The device of claim 1, wherein the controller aligns featuresin the first image with corresponding features in the second image toobtain the third image.
 11. A method comprising: operating, by acomputing device, a first camera to capture, via a first aperture at afirst location, a first image of a field of view, wherein the firstimage is a grayscale image; operating, by the computing device, a secondcamera, comprising a color filter, that captures, via a second apertureat a second location, a second image of the field of view, wherein thesecond image is a color image including a representation of a colorcontent of the field of view at multiple different colors orwavelengths, and wherein the first location and second location areseparated by a distance such that the contents of the first image andsecond image differ due to the parallax effect; determining, by thecomputing device, a luminance content of the second image; based on theluminance content of the second image being greater than a threshold,incorporating, by the computing device, the color content obtained fromthe color image into the grayscale image obtained from the first camerato obtain a third image; and causing, by the computing device, a displayto display the third image.
 12. The method of claim 11, wherein thecolor filter includes at least one of a Bayer filter, an RGBE filter, aCYYM filter, a CYGM filter, or an RGBW filter.
 13. The method of claim11, wherein the grayscale image includes a representation of luminancecontent in the field of view, and wherein the third image includesluminance content of the first image and color content of the secondimage.
 14. The method of claim 11, wherein the first image has a firstpixel resolution indicative of a spatial resolution of the first image,and wherein the second image has a second pixel resolution indicative ofa spectral resolution of the second image.
 15. The method of claim 14,wherein the third image has a third pixel resolution indicative of thespatial resolution of the first image and the spectral resolution of thesecond image.
 16. The method of claim 14, wherein the first pixelresolution of the first image is greater than the second pixelresolution of the second image.
 17. The method of claim 16, wherein thethird image has a third pixel resolution that is substantially same asthe first pixel resolution of the first image.
 18. The method of claim11, further comprising: aligning, by the computing device, features inthe first image with corresponding features in the second image toobtain the third image.
 19. A non-transitory computer readable mediumstoring instructions that, when executed by one or more processors of acomputing device, cause the computing device to perform operationscomprising: causing a first camera to capture, via a first aperture at afirst location, a first image of a field of view, wherein the firstimage is a grayscale image; causing a second camera that comprises acolor filter to capture, via a second aperture at a second location, asecond image of the field of view, wherein the second image is a colorimage including a representation of a color content of the field of viewat multiple different colors or wavelengths, and wherein the firstlocation and second location are separated by a distance such that thecontents of the first image and second image differ due to the parallaxeffect; determine a luminance content of the second image; based on theluminance content of the second image being greater than a threshold,incorporating the color content obtained from the color image into thegrayscale image obtained from the first camera to obtain a third image;and causing a display to display the third image.
 20. The non-transitorycomputer readable medium of claim 19, wherein the operations furthercomprise: aligning features in the first image with correspondingfeatures in the second image to obtain the third image.